Image processing method and apparatus, and non-transitory computer readable storage medium

ABSTRACT

An image processing method and apparatus, and a non-transitory computer-readable storage medium are provided. The method includes: acquiring an exposure duration and at least one motion component corresponding to at least one direction within the exposure duration of a current video frame; performing a first noise reduction operation on the current video frame in a two-dimensional space domain in response to a first motion component of the at least one motion component being greater than a preset motion component threshold; and performing a second noise reduction operation on the current video frame in a three-dimensional space domain in response to the at least one motion component being less than the preset motion component threshold.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of International Patent ApplicationNo. PCT/CN2020/076670, filed Feb. 25, 2020, the entire disclosure ofwhich is incorporated herein by reference.

TECHNICAL FIELD

Embodiments of the present disclosure relate to the technical field ofimage processing, and more specifically, to an image processing method,an image processing apparatus, and a non-transitory computer-readablestorage medium.

BACKGROUND

At present, in order to reduce a noise in a video and improve an imagedisplay effect of the video, multi-frame noise reduction algorithm isused to reduce the noise of a video frame of the video. Specifically, aframe alignment is performed on a denoised previous video frame and acurrent video frame having noise, and then a fusion operation isperformed on aligned video frames to achieve the multi-frame noisereduction for the video frames.

However, an effect of a multi-frame alignment is poor when the mobilephone is moving fast, thereby leading to an unsatisfactory effect of themulti-frame noise reduction. Generally, an alignment accuracy betweentwo frames is increased by adding algorithms. However, adding thealgorithms will lead to a long time of image processing and a low speedof image processing.

SUMMARY

Embodiments of the present disclosure provide an image processingmethod, an image processing apparatus, and a non-transitorycomputer-readable storage medium.

Technical solutions in some embodiments of the present disclosure areachieved below.

An image processing method is provided in some embodiments of thepresent disclosure, the method includes: acquiring an exposure durationof a current video frame and at least one motion component correspondingto at least one direction within the exposure duration; performing afirst noise reduction operation on the current video frame in atwo-dimensional space domain in response to a first motion component ofthe at least one motion component being greater than a preset motioncomponent threshold; and performing a second noise reduction operationon the current video frame in a three-dimensional space domain inresponse to the at least one motion component being less than the presetmotion component threshold.

An image processing apparatus is further provided in some embodiments ofthe present disclosure. The image processing apparatus includes: aprocessor, a memory storing instructions executable by the processor, acommunication interface, and a bus configured to connect the processor,the memory, and the communication interface. When the instructions areexecuted, the processor performs above method, and the method comprises:acquiring an exposure duration of a current video frame and at least onemotion component corresponding to at least one direction within theexposure duration; performing a first noise reduction operation on thecurrent video frame in a two-dimensional space domain in response to afirst motion component of the at least one motion component beinggreater than a preset motion component threshold; and performing asecond noise reduction operation on the current video frame in athree-dimensional space domain in response to the at least one motioncomponent being less than the preset motion component threshold.

A non-transitory computer-readable storage medium is provided in someembodiments of the present disclosure. The non-transitorycomputer-readable storage medium stores a program applied to an imageprocessing apparatus, and the program is executed by a processor toimplement above method, and the method comprises: acquiring an exposureduration of a current video frame and at least one motion componentcorresponding to at least one direction within the exposure duration;performing a first noise reduction operation on the current video framein a two-dimensional space domain in response to a first motioncomponent of the at least one motion component being greater than apreset motion component threshold; and performing a second noisereduction operation on the current video frame in a three-dimensionalspace domain in response to the at least one motion component being lessthan the preset motion component threshold.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 (a) is a schematic diagram illustrating a current video frameshot by a shooting device in the related art when the shooting device ismoving by a small distance.

FIG. 1 (b) is a schematic diagram illustrating a current video frameshot by the shooting device in the related art when the shooting deviceis moving quickly.

FIG. 2 is a flowchart of an image processing method according to someembodiments of the present disclosure.

FIG. 3 is a schematic diagram illustrating an exemplary image processingdevice acquiring angular velocity components of a gyroscope in threedirections of X, Y, and Z according to some embodiments of the presentdisclosure.

FIG. 4 is a schematic diagram of an exemplary image processing deviceacquiring motion radians of the gyroscope in the three directions of X,Y, and Z according to some embodiments of the present disclosure.

FIG. 5 is a schematic flowchart of an exemplary image processing methodaccording to some embodiments of the present disclosure.

FIG. 6 is a displayed diagram illustrating that no “ghost” exists in thecurrent video frame shot by a shooting device according to someembodiments of the present disclosure when the shooting device is movedquickly.

FIG. 7 is a first structural block diagram of an image processingapparatus according to some embodiments of the present disclosure.

FIG. 8 is a second structural block diagram of an image processingapparatus according to some embodiments of the present disclosure.

DETAILED DESCRIPTION

The technical solutions in some embodiments of the present disclosurewill be clearly and completely described below in conjunction withdrawings in the embodiments of the present disclosure. It should beunderstood that the embodiments described here are only used to explainthe related disclosure, but not to limit the disclosure. In addition, itshould be noted that, for ease of description, only parts related to thepresent disclosure are shown in the drawings.

When a still photo is shot or a shooting device is fixed, a mobile phonecan be kept as still as possible since a shooting time is short. Thus,in this case, a camera may be moved by a small distance, and lessserious “ghost” may be generated in multi-frame noise reduction fusion.As shown in FIG. 1 (a), the shooting device is moving by a smalldistance or having a small motion amplitude when shooting or capturing acurrent video frame, in this case, the current video frame captured mayhave less “ghost”. Herein, the terms “motion amplitude” of a devicemeans the moving degree (distance and/or angle) of the device. As shownin FIG. 1(b), in case that a user is holding the shooting device toshoot a video such that it is difficult for the user to maintain aposture within dozens of seconds, or in case that the user enormouslyand dynamically moves the shooting device to capture various dynamicscenes, motion vectors of a previous frame and a next frame may varygreatly, such that the alignment accuracy is insufficient, therebygenerating serious “ghost” during a synthesis of last two frames. Inaddition, a video frame having the “ghost” will continue to be used fora synthesis of a next video frame, such that the “ghost” will be presentin all subsequent video frames, which affects an effect of the noisereduction of the video. In order to solve above problems, technicalsolutions of the present disclosure may be provided.

The technical solutions in the embodiments of the present disclosurewill be clearly and completely described below in conjunction with thedrawings in the embodiments of the present disclosure.

In some embodiments of the present disclosure, an image processingmethod may be provided. FIG. 2 is a flowchart of the image processingmethod according to some embodiments of the present disclosure. Themethod may include following operations.

In operation S101, an exposure duration of a current video frame and atleast one motion component corresponding to at least one directionwithin the exposure duration may be acquired.

The image processing method provided in some embodiments of the presentdisclosure is applicable to a scene where a multi-frame noise reductionoperation is performed for video frames in an acquired or capturedvideo.

In some embodiments, an image processing apparatus may be any apparatuswith image acquisition and image processing functions, such as a tabletcomputer, a mobile phone, a personal computer (PC), a notebook computer,a camera, a network TV, a wearable apparatus, and so on.

In some embodiments, the image processing apparatus may acquire theexposure duration of the current video frame in response to the imageprocessing apparatus receiving the current video frame to be processed.Specifically, the image processing apparatus may acquire a Start ofFrame (SOF) Timestamp and an End of Frame (EOF) Timestamp from ametadata of the current video frame. After that, the image processingapparatus may determine a duration between the EOF and the SOF as theexposure duration of the current video frame.

In some embodiments, the image processing apparatus may acquire at leastone motion component of the image processing apparatus in orcorresponding to at least one direction within the exposure duration.The motion component may be an angular velocity component or anacceleration component, etc., which may be specifically selectedaccording to actual conditions, and which may not be specificallylimited in some embodiments of the present disclosure.

In some embodiments, the image processing apparatus may include anangular velocity sensor. The image processing apparatus may acquire anangular velocity component in at least one dimension from the angularvelocity sensor within the exposure duration. The image processingapparatus may determine the angular velocity component in the at leastone dimension to be the at least one motion component corresponding tothe at least one direction. Specifically, the image processing apparatusmay divide the exposure duration into a set of time samples. The imageprocessing apparatus may acquire at least one set ofangular-velocity-component samples in the least one dimension within orduring the set of time samples from the angular velocity sensor, andeach of the at least one dimension corresponds to one of the at leastone set of angular-velocity-component samples within or during the setof time samples. Then, the image processing apparatus may determine theangular velocity component in the at least one dimension within theexposure duration according to or based on the at least one set ofangular-velocity-component samples, and determine the angular velocitycomponent in the at least one dimension as the at least one motioncomponent corresponding to the at least one direction.

In some embodiments, the angular velocity sensor may be a deviceconfigured to perform an angular velocity measurement, such as agyroscope. The number of the gyroscopes may be determined according tothe at least one dimension acquired based on actual needs, which is notspecifically limited in some embodiments of the present disclosure.

In some embodiments, the angular velocity component in the at least onedimension may be a two-dimensional angular velocity component or athree-dimensional angular velocity component, etc., which may bespecifically selected according to actual conditions, and which may notbe specifically limited in some embodiments of the present disclosure.Taking the three-dimensional angular velocity component as an example,the image processing apparatus may use the gyroscope to collect anX-axis angular velocity component, a Y-axis angular velocity component,and a Z-axis angular velocity component within the exposure duration.

It should be noted that, the angular velocity component in the at leastone dimension is configured to represent or indicate whether the imageprocessing apparatus is moving quickly. Therefore, in response to theimage processing apparatus acquiring the angular velocity component inthe at least one dimension within the exposure duration, the imageprocessing apparatus may determine a degree of motion or the motionamplitude thereof based on an integral of an absolute value of theangular velocity component within the exposure duration without needingto consider a specific direction of motion.

Exemplarily, as shown in FIG. 3 , the image processing apparatus maydetermine the duration between the SOF Timestamp and the EOF Timestampas the exposure duration. The image processing apparatus may divide theexposure duration into N time samples, and acquire angular velocitycomponents of the gyroscope in three directions of X, Y, and Z duringeach time sample of the N time samples. The gyroscope may be configuredto calibrate angles in the three directions of X, Y, and Z throughCartesian coordinates. In this way, the image processing apparatus mayacquire a set of angular velocity components X₀, X₁, . . . X_(N) of thegyroscope in the X axis within the exposure duration. The imageprocessing apparatus may determine the angular velocity component of theimage processing apparatus in the X axis by using the set of angularvelocity components X₀, X₁, . . . , X_(N), which is specifically shownin formula (1).

$\begin{matrix}{{{angular}{velocity}{component}{of}X{axis}} = {\sum\limits_{N = 0}^{N = {n - 1}}{❘X_{N}❘}}} & (1)\end{matrix}$

The image processing apparatus may acquire a set of angular velocitycomponents Y₀, Y₁, . . . , Y_(N) of the gyroscope in the Y axis withinthe exposure duration. The image processing apparatus may determine theangular velocity component of the image processing apparatus in the Yaxis through or by using the set of angular velocity components Y₀, Y₁,. . . , Y_(N), which is specifically shown in formula (2).

$\begin{matrix}{{{angular}{velocity}{component}{of}Y{axis}} = {\sum\limits_{N = 0}^{N = {n - 1}}{❘Y_{N}❘}}} & (2)\end{matrix}$

The image processing apparatus may acquire a set of angular velocitycomponents Z₀, Z₁, . . . , Z_(N) of the gyroscope in the Z axis withinthe exposure duration. The image processing apparatus may determine theangular velocity component of the image processing apparatus in the Zaxis through or by using the set of angular velocity components Z₀, Z₁,. . . , Z_(N), which is specifically shown in formula (3).

$\begin{matrix}{{{angular}{velocity}{component}{of}Z{axis}} = {\sum\limits_{N = 0}^{N = {n - 1}}{❘Z_{N}❘}}} & (3)\end{matrix}$

In operation S102, in response to a first motion component of the atleast one motion component being greater than a preset motion componentthreshold, a first noise reduction operation may be performed on thecurrent video frame in a two-dimensional space domain. The first noisereduction operation may be an intraframe noise reduction operationperformed on the current video frame without using a previous videoframe prior to the current video frame and a next video frame next tothe current video frame.

After the image processing apparatus acquires the exposure duration andthe at least one motion component corresponding to the at least onedirection within the exposure duration, the image processing apparatusmay compare the at least one motion component with the preset motioncomponent threshold. In response to the first motion component of the atleast one motion component being greater than the preset motioncomponent threshold, the image processing apparatus may perform thefirst noise reduction operation on the current video frame in thetwo-dimensional space domain.

In some embodiments, the image processing apparatus may be preset withthe preset motion component threshold, and the image processingapparatus may compare the at least one motion component with the presetmotion component threshold. In response to the first motion componentgreater than the preset motion component threshold existing in the atleast one motion component, it indicates that the motion amplitude ofthe image processing apparatus is too great or overlarge when the imageprocessing apparatus is capturing the current video frame. In this case,the image processing apparatus may perform the first noise reductionoperation on the current video frame in the two-dimensional spacedomain, and prohibit or disable the current video frame from being fusedwith an adjacent historical video frame and an adjacent next videoframe. It should be noted that the adjacent historical video frame isthe previous video frame prior to the current video frame, and theadjacent next video frame is next to the current video frame.

In some embodiments, the image processing apparatus may perform thefirst noise reduction operation on the current video frame in thetwo-dimensional space domain through noise reduction methods, such asGaussian filter, Bilateral filter, or the like.

In some embodiments, in response to the image processing apparatusdetermining that the first motion component is greater than the presetmotion component threshold, in order to improve an accuracy of detectingthe motion of the image processing apparatus, the image processingapparatus may further acquire at least one motion radian correspondingto the at least one direction within the exposure duration, and comparethe at least one motion radian with a preset radian threshold. Inresponse to the image processing apparatus determining that a firstmotion radian of the at least one motion radian is greater than thepreset radian threshold, the image processing apparatus may perform thefirst noise reduction operation on the current video frame in thetwo-dimensional space domain.

Exemplarily, as shown in FIG. 4 , the image processing apparatus mayacquire angular velocity components of the gyroscope in the threedirections along the X axis, Y axis, and Z axis during N time samples.Then, the image processing apparatus may acquire a set of angularvelocity components X₀, X₁, . . . , X_(N) of the gyroscope in or alongthe X axis within or during time samples t₀, t₁, . . . , t_(N). Theimage processing apparatus may determine the motion radian of the imageprocessing apparatus along the X axis by using the set of angularvelocity components X₀, X₁, . . . , X_(N), which is specifically shownin formula (4).

$\begin{matrix}{{{motion}{radian}{of}X{axis}} = {\sum\limits_{N = 1}^{N = {n - 1}}{{❘X_{N}❘} \times \left( {t_{N} - t_{N - 1}} \right)}}} & (4)\end{matrix}$

The image processing apparatus may acquire a set of angular velocitycomponents Y₀, Y₁, . . . , Y_(N) of the gyroscope along the Y axisduring time samples t₀, t₁, . . . , t_(N). The image processingapparatus may determine the motion radian of the image processingapparatus along the Y axis by using the set of angular velocitycomponents Y₀, Y₁, . . . , Y_(N), which is specifically shown in formula(5).

$\begin{matrix}{{{motion}{radian}{of}Y{axis}} = {\sum\limits_{N = 1}^{N = {n - 1}}{{❘Y_{N}❘} \times \left( {t_{N} - t_{N - 1}} \right)}}} & (5)\end{matrix}$

The image processing apparatus may acquire a set of angular velocitycomponents Z₀, Z₁, . . . , Z_(N) of the gyroscope along the Z axisduring time samples t₀, t₁, . . . , t_(N). The image processingapparatus may determine the motion radian of the image processingapparatus along the Z axis by using the set of angular velocitycomponents Z₀, Z₁, . . . , Z_(N), which is specifically shown in formula(6).

$\begin{matrix}{{{motion}{radian}{of}Z{axis}} = {\sum\limits_{N = 1}^{N = {n - 1}}{{❘Z_{N}❘} \times \left( {t_{N} - t_{N - 1}} \right)}}} & (6)\end{matrix}$

In operation S103, in response to the at least one motion componentbeing less than the preset motion component threshold, a multi-framenoise reduction operation may be performed on the current video frame ina three-dimensional space domain. Herein, the multi-frame noisereduction operation may also be referred as a second noise reductionoperation in the following, and the multi-frame noise reductionoperation may be an interframe noise reduction operation during whichthe current video frame may be fused with the previous video frame andthe next video frame.

After the image processing apparatus acquires the exposure duration andthe at least one motion component corresponding to the at least onedirection within the exposure duration of the current video frame, theimage processing apparatus may compare the at least one motion componentwith the preset motion component threshold. In response to the imageprocessing apparatus determining that all of the at least one motioncomponent is less than the preset motion component threshold, the imageprocessing apparatus may perform the multi-frame noise reductionoperation on the current video frame in the three-dimensional spacedomain.

Exemplarily, the image processing apparatus may acquire the motioncomponents in the X, Y, and Z three directions of the gyroscope withinthe exposure duration, and compare the motion components in the X, Y,and Z directions with the preset motion component threshold. In responseto the motion components in the X, Y, and Z three directions being allless than the preset motion component threshold, it indicates that themotion amplitude of the image processing apparatus satisfies a conditionof the multi-frame noise reduction operation when the image processingapparatus is collecting the current video frame. In this case, the imageprocessing apparatus may perform the multi-frame noise reductionoperation on the current video frame in the three-dimensional spacedomain.

In some embodiments, in order to improve the accuracy of detecting themotion of the image processing apparatus, the image processing apparatusmay further acquire the at least one motion radian corresponding to theat least one direction within the exposure duration, and compare the atleast one motion radian with the preset radian threshold. In response tothe image processing apparatus determining that the first motion radianof the at least one motion radian is less than the preset radianthreshold, the image processing apparatus may align the current videoframe with the adjacent historical video frame and the adjacent nextvideo frame, thereby performing the multi-frame noise reductionoperation on the current video frame.

Exemplarily, as shown in FIG. 5 , the flow chart for the imageprocessing apparatus to determine whether to perform the first noisereduction operation in the two-dimensional space domain on the currentframe according to the motion amplitude of the image processingapparatus within the exposure duration is described below.

1. The image processing apparatus may synchronize the exposure durationof the current video frame and angular velocity data of the gyroscope inthe X, Y, and Z directions within the exposure duration. The X directionis substantially perpendicular to the Y direction and the Z direction,and the Y direction is substantially perpendicular to the Z direction.

2. The image processing apparatus may acquire motion velocity values inthe X, Y, and Z three directions according to the angular velocity dataof the gyroscope in the X, Y, and Z directions within the exposureduration.

3. The image processing apparatus may compare the motion velocity valuesin the X, Y, and Z directions with a preset motion velocity valuethreshold.

4. In response to the motion velocity values in the X, Y, and Zdirections being all less than the preset motion velocity valuethreshold, an operation 9 may be executed.

5. In response to at least one of the motion velocity values in the X,Y, and Z directions among the motion velocity values being greater thanthe preset motion velocity value threshold, the image processingapparatus may acquire motion angles in the X, Y, and Z directions.

6. The image processing apparatus may compare the motion angles in theX, Y, and Z three directions with a preset motion angle threshold.

7. In response to the motion angles in the X, Y, and Z directions beingall less than the preset motion angle threshold, the operation 9 may beexecuted.

8. In response to at least one motion angle of the motion angles in theX, Y, and Z directions being greater than the preset motion anglethreshold, an operation 10 may be executed.

9. The image processing apparatus may perform a 3D interframe noisereduction fusion operation on the current video frame, the adjacentprevious video frame prior to the current video frame, and the adjacentnext video frame next to the current video frame. That is to say, thecurrent video frame may be fused with the adjacent previous video frame,and further fused with the adjacent next video frame.

10. The image processing apparatus may perform the 2D intraframe noisereduction operation on the current video frame in the 2D space domain.

It can be understood that, when the image processing apparatus isperforming the noise reduction on the current video frame, the imageprocessing apparatus may first acquire the at least one motion componentin the at least one direction of the image processing apparatus withinthe exposure duration of the current video frame, and then determine themotion component of the image processing apparatus based on the at leastone motion component within the exposure duration. In response to the atleast one motion component of the image processing apparatus within theexposure duration being greater than the preset motion componentthreshold, the image processing apparatus may directly perform the firstnoise reduction operation on the current video frame in thetwo-dimensional space domain to reduce the operations for processing thealignment accuracy of multiple frames, thereby shortening the timeduration of an image processing, and improving the speed of the imageprocessing. In some embodiments, as shown in FIG. 6 , when the imageprocessing apparatus is moving quickly, the image processing apparatusmay only perform the 2D noise reduction operation on the current videoframe, and may not perform the fusion of the previous frame and the nextframe, thereby reducing the occurrence of the “ghost”.

Based on above-mentioned embodiments, in some other embodiments of thepresent disclosure, FIG. 7 is a first schematic diagram of aconfiguration structure of the image processing apparatus according tosome embodiments of the present disclosure. As shown in FIG. 7 , theimage processing apparatus 1 according to some embodiments of thepresent disclosure may include an acquisition part 10, a two-dimensionalnoise reduction part 11, and a multi-frame noise reduction part 12.

The acquisition part 10 is configured to acquire an exposure duration ofa current video frame and at least one motion component corresponding toat least one direction within the exposure duration.

The two-dimensional noise reduction part 11 is configured to perform afirst noise reduction operation on the current video frame in atwo-dimensional space domain in response to a first motion component ofthe at least one motion component being greater than a preset motioncomponent threshold.

The multi-frame noise reduction part 12 is configured to perform amulti-frame noise reduction operation on the current video frame in athree-dimensional space domain in response to the at least one motioncomponent being less than the preset motion component threshold.

Further, the image processing apparatus 1 includes a calculation part.

The calculation part is configured to acquire at least one motion radiancorresponding to the at least one direction within the exposure durationin response to the first motion component being greater than the presetmotion component threshold.

The two-dimensional noise reduction part 11 is further configured toperform the first noise reduction operation on the current video framein the two-dimensional space domain in response to a first motion radianof the at least one motion radian being greater than a preset radianthreshold.

Further, the image processing apparatus includes an angular velocitysensor. The image processing apparatus further includes a division partand a determination part.

The division part is configured to divide the exposure duration into aset of time samples.

The acquisition part 10 is configured to acquire at least one set ofangular-velocity-component samples in at least one dimension during theset of time samples from the angular velocity sensor, and each of the atleast one dimension corresponds to one of the at least one set ofangular-velocity-component samples during the set of time samples.

The determination part is configured to determine an angular velocitycomponent in the at least one dimension within the exposure durationbased on the at least one set of angular-velocity-component samples. Thedetermination part is also configured to determine the angular velocitycomponent in the at least one dimension as the at least one motioncomponent corresponding to the at least one direction.

Further, the image processing apparatus includes a prohibition part.

The prohibition part is configured to prohibit or disable the currentvideo frame from being fused with an adjacent historical video frame andan adjacent next video frame in response to the at least one motioncomponent being greater than the preset motion component thresholdand/or the at least one motion radian being greater than the presetradian threshold. In other words, the prohibition part may be configuredto prohibit the current video frame from being fused with an adjacenthistorical video frame and an adjacent next video frame in response tosatisfying at least one following condition: the at least one motioncomponent is greater than the preset motion component threshold, and theat least one motion radian being greater than the preset radianthreshold. It should be noted that, the term “A and/or B” includes Aexisting alone, B existing alone, and A and B existing simultaneously.

Further, the multi-frame noise reduction part 12 is configured toperform the multi-frame noise reduction operation on the current videoframe by performing the multi-frame alignment on the current video framewith the adjacent historical video frame and the adjacent next videoframe in response to the first motion component being less than thepreset motion component threshold and/or the first motion radian beingless than the preset radian threshold.

FIG. 8 is a second schematic diagram of the composition structure of theimage processing apparatus according to some embodiments of thedisclosure. As shown in FIG. 8 , the image processing apparatus 1 mayfurther include a processor 110, a memory 111 storing instructionsexecutable by the processor 110, a communication interface 112, and abus 113 configured to connect the processor 110, the memory 111, and thecommunication interface 112.

In some embodiments of the present disclosure, the processor 110 may beat least one of an Application Specific Integrated Circuit (ASIC), aDigital Signal Processor (DSP), a Digital Signal Processing Device(DSPD), a Programmable Logic Device (PLD), a Field Programmable GateArray (FPGA), a Central Processing Unit (CPU), a controller, amicrocontroller, and a microprocessor. It should be understood that, fordifferent devices, electronic components configured to implement theabove-mentioned functions of the processor may also be other electroniccomponents, which is not specifically limited in some embodiments of thepresent disclosure. The image processing apparatus 1 may also includethe memory 111 connected to the processor 110, and the memory 111 isconfigured to store executable program codes including computeroperation instructions. The memory 111 may include a high-speed RandomAccess Memory (RAM) or a non-volatile memory, for example, at least twodisk memories.

In some embodiments, the bus 113 is configured to connect thecommunication interface 112, the processor 110, and the memory 111. Thebus 113 is also configured to achieve the communication among thesecomponents.

In some embodiments, the memory 111 is configured to store theinstructions and data.

Further, in some embodiments, the processor 110 is configured to acquirethe exposure duration of the current video frame and the at least onemotion component corresponding to the at least one direction within theexposure duration. In response to the first motion component of the atleast one motion component being greater than the preset motioncomponent threshold, the processor 110 is configured to perform thefirst noise reduction operation on the current video frame in thetwo-dimensional space domain. In response to the at least one motioncomponent being less than the preset motion component threshold, theprocessor 110 is configured to perform the multi-frame noise reductionoperation on the current video frame in the three-dimensional spacedomain.

Further, in some embodiments, the processor 110 is configured to acquirethe at least one motion radian corresponding to the at least onedirection within the exposure duration in response to the first motioncomponent being greater than the preset motion component threshold. Inresponse to the first motion radian of the at least one motion radianbeing greater than a preset radian threshold, the processor 110 isconfigured to perform the first noise reduction operation on the currentvideo frame in the two-dimensional space domain.

Further, in some embodiments, the processor 110 is configured to acquirethe angular velocity component in at least one dimension within theexposure duration from the angular velocity sensor. The processor 110 isconfigured to determine the angular velocity component in the at leastone dimension as the at least one motion component corresponding to theat least one direction.

Further, in some embodiments, the processor 110 is configured toprohibit or disable the current video frame from being fused with theadjacent historical video frame and the adjacent next video frame inresponse to the at least one motion component being greater than thepreset motion component threshold and/or the at least one motion radianbeing greater than the preset radian threshold.

Further, in some embodiments, the processor 110 is configured to performthe multi-frame noise reduction operation on the current video frame byperforming the multi-frame alignment on the current video frame with theadjacent historical video frame and the adjacent next video frame inresponse to the first motion component being less than the preset motioncomponent threshold and/or the first motion radian being less than thepreset radian threshold.

In practical applications, the memory 111 may be a volatile memory, suchas a Random-Access Memory (RAM). The memory 111 may also be anon-volatile memory, such as a Read-Only Memory (ROM), a flash memory, aHard Disk Drive (HDD), a Solid-State Drive (SSD). Or, the memory 111 maybe a combination of the above-mentioned types of memories, and thememory 111 may be configured to provide instructions and data to theprocessor 110.

In addition, in some embodiments of the present disclosure, all functionmodules may be integrated into one processing unit. Alternatively, eachunit may exist alone physically, or two or more units may be integratedinto one unit. The above-mentioned integrated unit may be implemented inform of a hardware or a software function module.

When the integrated unit is implemented in form of the software functionmodule and is not sold or used as an independent product, the integratedunit may be stored in a non-transitory computer-readable storage medium.In this way, the technical solution of some embodiments may beessentially embodied in form of a software product. Alternatively, apart that contributes to the related technology may be embodied in formof the software product. Alternatively, all or part of the technicalsolution may be embodied in form of the software product. The computersoftware product is stored in the non-transitory computer-readablestorage medium including several instructions to enable a computerdevice (such as a personal computer, a server, or a network device,etc.) or a processor to execute all or part of operations of the methodsin some embodiments of the present disclosure. The computer-readablestorage medium includes various kinds of medium storing program codessuch as a U disk, a mobile hard disk, a Read Only Memory (ROM), a RandomAccess Memory (RAM), a magnetic disk or an optical disk.

Some embodiments of the present disclosure discloses an image processingmethod, and the method includes: acquiring an exposure duration of acurrent video frame and at least one motion component corresponding toat least one direction within the exposure duration; performing a firstnoise reduction operation on the current video frame in atwo-dimensional space domain in response to a first motion component ofthe at least one motion component being greater than a preset motioncomponent threshold; and performing a second noise reduction operationon the current video frame in a three-dimensional space domain inresponse to the at least one motion component being less than the presetmotion component threshold.

In some embodiments, the performing a noise reduction operation on thecurrent video frame in a two-dimensional space domain in response to afirst motion component of the at least one motion component beinggreater than a preset motion component threshold, includes: acquiring atleast one motion radian corresponding to the at least one direction inresponse to the first motion component being greater than the presetmotion component threshold; and performing the first noise reductionoperation on the current video frame in the two-dimensional space domainin response to a first motion radian of the at least one motion radianbeing greater than a preset radian threshold.

In some embodiments, the acquiring at least one motion componentcorresponding to at least one direction within the exposure duration,includes: dividing the exposure duration into a set of time samples;acquiring at least one set of angular-velocity-component samples in atleast one dimension during the set of time samples, wherein each of theat least one dimension corresponds to one of the at least one set ofangular-velocity-component samples during the set of time samples;determining an angular velocity component in the at least one dimensionwithin the exposure duration based on the at least one set ofangular-velocity-component samples; and determining the angular velocitycomponent in the at least one dimension as the at least one motioncomponent corresponding to the at least one direction.

In some embodiments, the method further includes: prohibiting thecurrent video frame from being fused with an adjacent historical videoframe and an adjacent next video frame in response to satisfying atleast one following condition: the at least one motion component beinggreater than the preset motion component threshold; and the at least onemotion radian being greater than the preset radian threshold.

In some embodiments, the method further includes: performing the secondnoise reduction operation on the current video frame by performing amulti-frame alignment on the current video frame with an adjacenthistorical video frame and an adjacent next video frame in response tosatisfying at least one following condition: the first motion componentbeing less than the preset motion component threshold; and the firstmotion radian being less than the preset radian threshold.

In some embodiments, the performing a second noise reduction operationon the current video frame in a three-dimensional space domain inresponse to the at least one motion component being less than the presetmotion component threshold further includes: performing the second noisereduction operation on the current video frame by aligning the currentvideo frame with an adjacent historical video frame and an adjacent nextvideo frame in response to all of the at least one motion componentbeing less than the preset motion component threshold.

In some embodiments, the at least one motion component includes anangular velocity component and an acceleration component.

In some embodiments, the method further includes: in response to the atleast one motion component including the angular velocity component inat least one dimension, determining a motion amplitude within theexposure duration based on an integral of an absolute value of theangular velocity component.

In some embodiments, the acquiring an exposure duration of a currentvideo frame and at least one motion component corresponding to at leastone direction within the exposure duration, includes: synchronizing theexposure duration of the current video frame and angular velocity datain first, second, and third directions within the exposure duration,wherein the first direction is substantially perpendicular to the seconddirection and the third direction, and the second direction issubstantially perpendicular to the third direction; and acquiring motionvelocity values in the first, second, and third directions according tothe angular velocity data in the first, second, and third directionswithin the exposure duration.

In some embodiments, the performing a first noise reduction operation onthe current video frame in a two-dimensional space domain in response toa first motion component of the at least one motion component beinggreater than a preset motion component threshold, includes: acquiringmotion angles in the first, second, and third directions in response toat least one of the motion velocity values among the motion velocityvalues in the first, second, and third directions being greater than apreset motion velocity value threshold; and performing the first noisereduction operation on the current video frame in the two-dimensionalspace domain in response to at least one of the motion angles in thefirst, second, and third directions being greater than a preset motionangle threshold.

The image processing apparatus according to some embodiments of thepresent disclosure may be configured to acquire the exposure duration ofthe current video frame and the at least one motion componentcorresponding to the at least one direction within the exposureduration. In response to the first motion component of the at least onemotion component being greater than the preset motion component, theimage processing apparatus is configured to perform the first noisereduction operation on the current video frame in the two-dimensionalspace domain. In response to the at least one motion component beingless than the preset motion component threshold, the image processingapparatus is configured to perform the multi-frame noise reductionoperation on the current video frame in the three-dimensional spacedomain. It can be seen that, in some embodiment of the presentdisclosure, when the image processing apparatus is performing the noisereduction on the current video frame, the image processing apparatus mayfirst acquire the at least one motion component in the at least onedirection of the image processing apparatus within the exposureduration, and then determine the motion value of the image processingapparatus based on the at least one motion component within the exposureduration. In response to the at least one motion component of the imageprocessing apparatus within the exposure duration being greater than thepreset motion component threshold, the image processing apparatusdirectly performs the first noise reduction operation on the currentvideo frame in the two-dimensional space domain to reduce the operationsof the processing of the alignment accuracy of multiple frames, therebyshortening the time of an image processing, and improving the speed ofthe image processing.

Some embodiments of the present disclosure further disclose an imageprocessing apparatus and the image processing apparatus includes: aprocessor, a memory storing instructions executable by the processor, acommunication interface, and a bus configured to connect the processor,the memory, and the communication interface, wherein when theinstructions are executed, the processor performs the above-mentionedimage processing methods.

Some embodiments of the present disclosure provide a non-transitorycomputer-readable storage medium storing a program, and the program isexecuted by the processor to implement the above-mentioned imageprocessing methods.

Specifically, in some embodiments, the program instructionscorresponding to the image processing method may be stored in anon-transitory computer-readable storage medium such as an optical disk,a hard disk, a USB flash drives, etc. In response to the programinstructions corresponding to the image processing method stored in thenon-transitory computer-readable storage medium being read or executedby an electronic device, the image processing methods described abovemay be implemented.

Those skilled in the art should understand that some embodiments of thepresent disclosure may provide a method, a system, or a computer programproduct. Therefore, some embodiments of the present disclosure may be inform of hardware embodiments, software embodiments, or some embodimentscombining software with hardware. Moreover, some embodiments of thepresent disclosure may be in form of computer program productsimplemented on one or more computer-usable storage media (including butnot limited to a disk storage, an optical storage, etc.) storingcomputer-usable program codes.

The present disclosure is described with reference to schematicflowcharts and/or block diagrams for implementing operations of methods,equipment (systems), and computer program products according to someembodiments of the present disclosure. It should be understood that eachoperation and/or block in the schematic flowcharts and/or block diagramsand a combination of the operations and/or blocks in the schematicflowcharts and/or block diagrams may be implemented by computer programinstructions. The computer program instructions may be provided to theprocessor of a general-purpose computer, a special-purpose computer, anembedded processor, or other programmable data processing devices togenerate a machine, so that the instructions executed by the processorof the computer or other programmable data processing devices areconfigured to generate an apparatus for implementing specified functionsin one or more operations in the schematic flowcharts and/or one or moreblocks in the block diagrams.

The computer program instructions may also be stored in acomputer-readable memory that may direct a computer or otherprogrammable data processing devices to work in a specific manner, sothat the instructions stored in the computer-readable memory produce aproduct including an instruction device. The instruction deviceimplements the specified functions in one or more processes in theschematic flowcharts and/or one or more blocks in the block diagrams.

The computer program instructions may also be loaded on a computer orother programmable data processing devices, so that a series ofoperation blocks are executed on the computer or the other programmableequipment to produce computer-implemented operations. In this way, theinstructions executed on the computer or other programmable equipmentmay implement specified functions in one or more processes in theschematic flowcharts and/or one or more blocks in the block diagrams.

The foregoing description are only some embodiments of the presentdisclosure, and are not used to limit the protection scope of thepresent disclosure.

INDUSTRIAL APPLICABILITY

The image processing apparatus is provided in some embodiments of thepresent disclosure. When the image processing apparatus is performingthe noise reduction on the current video frame, the image processingapparatus first acquires the at least one motion component in at leastone direction of the image processing apparatus within the exposureduration, and then determines the motion amplitude of the imageprocessing apparatus based on the at least one motion component withinthe exposure duration. In response to the at least one motion componentof the image processing apparatus within the exposure duration beinggreater than the preset motion component threshold, the image processingapparatus directly performs the first noise reduction operation on thecurrent video frame in the two-dimensional space domain to reduce theoperations of the processing of the alignment accuracy of multipleframes, thereby shortening the time duration of the image processing,and improving the speed of the image processing.

What is claimed is:
 1. An image processing method, comprising: acquiringan exposure duration of a current video frame and at least one motioncomponent corresponding to at least one direction within the exposureduration; performing a first noise reduction operation on the currentvideo frame in a two-dimensional space domain in response to a firstmotion component of the at least one motion component being greater thana preset motion component threshold; and performing a second noisereduction operation on the current video frame in a three-dimensionalspace domain in response to the at least one motion component being lessthan the preset motion component threshold.
 2. The image processingmethod of claim 1, wherein performing the first noise reductionoperation on the current video frame in the two-dimensional space domainin response to the first motion component of the at least one motioncomponent being greater than the preset motion component threshold,comprises: acquiring at least one motion radian corresponding to the atleast one direction in response to the first motion component beinggreater than the preset motion component threshold; and performing thefirst noise reduction operation on the current video frame in thetwo-dimensional space domain in response to a first motion radian of theat least one motion radian being greater than a preset radian threshold.3. The image processing method of claim 1, wherein acquiring at leastone motion component corresponding to at least one direction within theexposure duration, comprises: dividing the exposure duration into a setof time samples; acquiring at least one set ofangular-velocity-component samples in at least one dimension during theset of time samples, wherein each of the at least one dimensioncorresponds to one of the at least one set of angular-velocity-componentsamples during the set of time samples; determining an angular velocitycomponent in the at least one dimension within the exposure durationbased on the at least one set of angular-velocity-component samples; anddetermining the angular velocity component in the at least one dimensionas the at least one motion component corresponding to the at least onedirection.
 4. The image processing method of claim 2, furthercomprising: prohibiting the current video frame from being fused with anadjacent historical video frame and an adjacent next video frame inresponse to satisfying at least one of the following conditions: the atleast one motion component being greater than the preset motioncomponent threshold; and the at least one motion radian being greaterthan the preset radian threshold.
 5. The image processing method ofclaim 2, further comprising: performing the second noise reductionoperation on the current video frame by performing a multi-framealignment on the current video frame with an adjacent historical videoframe and an adjacent next video frame in response to satisfying atleast one of the following conditions: the first motion component beingless than the preset motion component threshold; and the first motionradian being less than the preset radian threshold.
 6. The imageprocessing method of claim 1, wherein performing the second noisereduction operation on the current video frame in the three-dimensionalspace domain in response to the at least one motion component being lessthan the preset motion component threshold further comprises: performingthe second noise reduction operation on the current video frame byaligning the current video frame with an adjacent historical video frameand an adjacent next video frame in response to all of the at least onemotion component being less than the preset motion component threshold.7. The image processing method of claim 1, wherein the at least onemotion component comprises an angular velocity component and anacceleration component.
 8. The image processing method of claim 7,further comprising: in response to the at least one motion componentcomprising the angular velocity component in at least one dimension,determining a motion amplitude within the exposure duration based on anintegral of an absolute value of the angular velocity component.
 9. Theimage processing method of claim 1, wherein acquiring the exposureduration of the current video frame and at least one motion componentcorresponding to at least one direction within the exposure duration,comprises: synchronizing the exposure duration of the current videoframe and angular velocity data in first, second, and third directionswithin the exposure duration, wherein the first direction issubstantially perpendicular to the second direction and the thirddirection, and the second direction is substantially perpendicular tothe third direction; and acquiring motion velocity values in the first,second, and third directions according to the angular velocity data inthe first, second, and third directions within the exposure duration.10. The image processing method of claim 9, wherein performing the firstnoise reduction operation on the current video frame in thetwo-dimensional space domain in response to the first motion componentof the at least one motion component being greater than the presetmotion component threshold, comprises: acquiring motion angles in thefirst, second, and third directions in response to at least one of themotion velocity values among the motion velocity values in the first,second, and third directions being greater than a preset motion velocityvalue threshold; and performing the first noise reduction operation onthe current video frame in the two-dimensional space domain in responseto at least one of the motion angles in the first, second, and thirddirections being greater than a preset motion angle threshold.
 11. Animage processing apparatus, comprising: a processor, a memory storinginstructions executable by the processor, a communication interface, anda bus configured to connect the processor, the memory, and thecommunication interface, wherein when the instructions are executed, theprocessor performs an image processing method, the method comprising:acquiring an exposure duration of a current video frame and at least onemotion component corresponding to at least one direction within theexposure duration; performing a first noise reduction operation on thecurrent video frame in a two-dimensional space domain in response to afirst motion component of the at least one motion component beinggreater than a preset motion component threshold; and performing asecond noise reduction operation on the current video frame in athree-dimensional space domain in response to the at least one motioncomponent being less than the preset motion component threshold.
 12. Theimage processing apparatus of claim 11, wherein performing the firstnoise reduction operation on the current video frame in thetwo-dimensional space domain in response to the first motion componentof the at least one motion component being greater than a preset motioncomponent threshold, comprises: acquiring at least one motion radiancorresponding to the at least one direction in response to the firstmotion component being greater than the preset motion componentthreshold; and performing the first noise reduction operation on thecurrent video frame in the two-dimensional space domain in response to afirst motion radian of the at least one motion radian being greater thana preset radian threshold.
 13. The image processing apparatus of claim11, wherein acquiring at least one motion component corresponding to atleast one direction within the exposure duration, comprises: dividingthe exposure duration into a set of time samples; acquiring at least oneset of angular-velocity-component samples in at least one dimensionduring the set of time samples, wherein each of the at least onedimension corresponds to one of the at least one set ofangular-velocity-component samples during the set of time samples;determining an angular velocity component in the at least one dimensionwithin the exposure duration based on the at least one set ofangular-velocity-component samples; and determining the angular velocitycomponent in the at least one dimension as the at least one motioncomponent corresponding to the at least one direction.
 14. The imageprocessing apparatus of claim 12, wherein the method further comprises:prohibiting the current video frame from being fused with an adjacenthistorical video frame and an adjacent next video frame in response tosatisfying at least one of the following conditions: the at least onemotion component being greater than the preset motion componentthreshold; and the at least one motion radian being greater than thepreset radian threshold.
 15. The image processing apparatus of claim 12,wherein the method further comprises: performing the second noisereduction operation on the current video frame by performing amulti-frame alignment on the current video frame with an adjacenthistorical video frame and an adjacent next video frame in response tosatisfying at least one of the following conditions: the first motioncomponent being less than the preset motion component threshold; and thefirst motion radian being less than the preset radian threshold.
 16. Anon-transitory computer-readable storage medium, storing a programapplied to an image processing apparatus, wherein the program isexecuted by a processor to implement an image processing method, themethod comprising: acquiring an exposure duration of a current videoframe and at least one motion component corresponding to at least onedirection within the exposure duration; performing a first noisereduction operation on the current video frame in a two-dimensionalspace domain in response to a first motion component of the at least onemotion component being greater than a preset motion component threshold;and performing a second noise reduction operation on the current videoframe in a three-dimensional space domain in response to the at leastone motion component being less than the preset motion componentthreshold.
 17. The non-transitory computer-readable storage medium ofclaim 16, wherein performing the first noise reduction operation on thecurrent video frame the two-dimensional space domain in response to thefirst motion component of the at least one motion component beinggreater than the preset motion component threshold, comprises: acquiringat least one motion radian corresponding to the at least one directionin response to the first motion component being greater than the presetmotion component threshold; and performing the first noise reductionoperation on the current video frame in the two-dimensional space domainin response to a first motion radian of the at least one motion radianbeing greater than a preset radian threshold.
 18. The non-transitorycomputer-readable storage medium of claim 16, wherein acquiring at leastone motion component corresponding to at least one direction within theexposure duration, comprises: dividing the exposure duration into a setof time samples; acquiring at least one set ofangular-velocity-component samples in at least one dimension during theset of time samples, wherein each of the at least one dimensioncorresponds to one of the at least one set of angular-velocity-componentsamples during the set of time samples; determining an angular velocitycomponent in the at least one dimension within the exposure durationbased on the at least one set of angular-velocity-component samples; anddetermining the angular velocity component in the at least one dimensionas the at least one motion component corresponding to the at least onedirection.
 19. The non-transitory computer-readable storage medium ofclaim 17, wherein the method further comprises: prohibiting the currentvideo frame from being fused with an adjacent historical video frame andan adjacent next video frame in response to satisfying at least one ofthe following conditions: the at least one motion component beinggreater than the preset motion component threshold; and the at least onemotion radian being greater than the preset radian threshold.
 20. Thenon-transitory computer-readable storage medium of claim 17, wherein themethod further comprises: performing the second noise reductionoperation on the current video frame by performing a multi-framealignment on the current video frame with an adjacent historical videoframe and an adjacent next video frame in response to satisfying atleast one of the following conditions: the first motion component isless than the preset motion component threshold; and the first motionradian is less than the preset radian threshold.